Adding Thresholds
Learn the intuition behind, and the application of, thresholds in finding anomalies.
In the last lesson, we detected a large number of “anomalies” with just one entry. This is very common in errors that do not happen very often. In our case, every once in a while, we get a 400 status code. However, the standard deviation is very low because it does not happen very often so even a single error can be considered way above the acceptable value.
We do not really want to receive an alert in the middle of the night just because of one 400 status code. We cannot have every curious developer fiddling with the dev tools in his browser wake us up in the middle of the night.
To eliminate rows with only a few entries, we set a threshold:
After eliminating potential anomalies with less than 10 entries, we get much fewer and probably more relevant results.
